Method and apparatus for quantitative 3-d imaging

ABSTRACT

Described is a method and apparatus for obtaining additional information from an object and a method for surface imaging and three-dimensional imaging. Single lens, single aperture, single sensor system and stereo optic systems may be modified in order to successfully generate surface maps of objects or three-dimensional representations of target objects. A variety of the aspects of the present invention provide examples of the use of an addressable pattern in order to overcome mismatching common to standard defocusing techniques.

PRIORITY CLAIM

The present application is a divisional application claiming the benefitof priority of U.S. Non-Provisional patent application Ser. No.12/011,016, filed Jan. 22, 2008, titled “Method and Apparatus ForQuantitative 3-D Imaging”; U.S. Provisional Patent Application No.60/881,776, filed Jan. 22, 2007, titled, “A Single-Lens 3-D ImagingDevice based on Defocusing Technique;” U.S. Provisional PatentApplication No. 60/881,779, filed Jan. 22, 2007, titled, “A Handheld 3-DMapping Device based on the Defocusing Technique;” and U.S. ProvisionalPatent Application No. 60/881,762, filed Jan. 22, 2007, titled, “Methodfor Reducing the Number of Viewpoints Necessary in Camera-BasedQuantitative 3-D Imaging Systems.”

BACKGROUND OF THE INVENTION

(1) Technical Field

The present invention is related to a quantitative visualization systemand, more specifically, to a system for obtaining additional informationfrom an object through surface and three-dimensional imaging.

(2) Background

Three-dimensional (3-D) imaging is a continuously evolving field thatcould benefit from improved imaging techniques. Enhanced 3-D imagingcould be used for a variety of purposes, such as to generatequantitative information about an imaged object (through quantitative3-D imaging). However, existing imaging techniques have failed tosufficiently support quantitative 3-D imagining. For example, when apoint that is not on the focal plane of an imaging system is imagedthrough the imaging system, the captured point detected by a sensor issaid to be defocused. If the imaging system has a large aperture, thenthe defocused point will appear blurred. For this reason, it has beensuggested that the blur of the image of a point can be used toquantitatively determine the distance from that point to the focal planein space. It has also been suggested that by knowing the position of thefocal plane, the imaging system could be used for quantitative 3-Dimaging. To reconstruct the 3-D position of a point, it is onlynecessary to measure the size and/or intensity of the blur disc (Z) andthe point position on the sensor (X, Y).

In practice, however, such a system is difficult to effectivelyimplement. First, a blurred image occupies a lot of space on the sensor,so sophisticated algorithms to separate overlapped images are necessary.Second, the amount of light entering the optical system does not changeappreciably (unless the focal plane is very close to the optical system)between a focused point and a defocused point. Thus, the blurred imageputs the same amount of energy onto the sensor as a focused image, butspread over a larger area. The intensity of a defocused image isinversely proportional to its area, so a quantitative measurement of thedistance between the focal plane and a point based only on blur requiresa sensor with an extremely high dynamic range. In real lenses, there isalso diffraction effects which make blurred images look more like ringsthan broad Gaussians in certain depth ranges, making the softwareprocessing side more complicated. See, for example, Wu, M.; Roberts, J.W.; and Buckley, M., “Three-dimensional fluorescent particle tracking atmicron-scale using a single camera,” Experiments in Fluids, 2005, 38,461-465. Even without lens aberrations or diffraction, image processingis complicated by the fact that since the depth information comes from ameasure of the diameter of a blur spot, the intensity of the imagedpoint affects the measurement. For example, if two defocused points Aand B have the same amount of defocus, but point A is brighter thanpoint B, typically point B's image will be measured as having a smallerdiameter than point A's simply because it does not rise as far from thebackground illumination in the scene.

The original “defocusing” concept recognized that in such a blur-basedsystem, the depth information is carried only by the marginal (outer)rays of the ray pencil that forms the image. See, for example, Willert,C. E.; and Gharib, M., “Three-dimensional particle imaging with a singlecamera,” Experiments in Fluids, 1992, 12, 353-358. It is the angle thatthese rays make with the sensor plane that dictates the sensitivity ofthe imaging system. Thus, an equivalent measurement should be possibleby placing small apertures offaxis in the imaging system, such that onlymarginal rays may pass through to form an image. If a blur system, asdescribed above, has its large aperture replaced with a small apertureplaced anywhere on the circumference of the large aperture, then theimage of a defocused point is now a small spot located on what wouldotherwise be the circumference of a blurred image. The end result isdepth information that is transmitted not by the size of a blurred spot,but rather by a lateral offset in a much smaller spot. Measuring thelocation of a spot on an image is much less sensitive to intensitydifferences than measuring its size.

The use of small apertures alleviates the dynamic range issues with ablurbased system, since the high f-number of the small aperture makesdiffraction blur (not defocus blur) the primary blurring agent in theimage. This means that within a large range of distances from the focalplane, the images are almost the same size.

Using off-axis apertures means that reconstruction of a point's positionin space now involves finding all the images of a single point on thesensor and measuring the distance between them. The images will appearin the same pattern as the aperture arrangement; for example, if threesmall apertures arranged as vertices of an equilateral triangle areused, then the image of a defocused point is three small spots arrangedin an equilateral triangle. The orientation of the images' trianglerelative to the apertures' triangle reveals whether the defocused pointis ahead of or in front of the focal plane.

Additionally, the size of the images' triangle relates to the distancebetween the defocused point and the focal plane. The size of thetriangle is 0 for a focused point which occurs when all three images areon top of each other. The size of the triangle increases as the amountof defocus increases. Multiple small images take up less space on thesensor than one large blurred one, so the overlap problem is alleviatedby this arrangement.

The matching problem in the reconstruction generates a new problem; ifthe object being imaged is a set of featureless points, then the imagesare indistinguishable and can only be matched according to theirrelative location (for example, finding all dots on an image that formequilateral triangles within some tolerance). This relatively loosematching criterion necessitates that three or more apertures be used toreduce the number of mismatches or “ghosts.”

A single off-axis aperture records depth information; however, Z cannotbe separated from the in-plane position of the point imaged. Twoapertures record the depth information and allow the in-plane positionto be extracted independently of Z. In practice, it is impossible toreconstruct a random point cloud with only two apertures because manyghost particles are generated when images are mismatched. Moreover, itis impossible to know if a particle was in front of or behind the focalplane from only two images. With three apertures, mismatches are reducedand the sign of the distance from the particle to the focal plane isknown by the orientation of the triangle formed by the images. See, forexample, Willert, C. E.; and Gharib, M., “Three-dimensional particleimaging with a single camera,” Experiments in Fluids, 1992, 12, 353-358.

The original practical implementation of the defocusing concept consistsof a single lens with three off-axis apertures imaging onto a singlemonochromatic sensor (i.e., three was deemed the minimum number ofapertures that produced acceptable results). It should be noted thatbecause the defocusing measurement is in fact a measurement of a point'sposition relative to the focal plane, it is necessary to know theposition of the device to know the absolute position of desired point.

The three off-axis apertures imaging onto a single monochromatic sensoralso has disadvantages. Overcrowding of the sensor is still an issuewhen the point density within the scene is high. In this case, eachpoint has up to three images on the sensor and there is still a possibledynamic range issue (i.e., a point on the focal plane will have threeimages that coincide on the sensor and thus will look three times asbright as defocused points). The dynamic range issue can be overcome byselectively illuminating the volume so that no points on the focal planeare imaged.

As described in U.S. Pat. Nos. 6,955,656 and 7,006,132, one solution tothe overcrowding problem is to image each aperture with a separatesensor. This adds to the matching criterion, because now each spot onthe image can only be one of the vertices of the aperture arrangement;since the source (aperture) of each spot is known, there is slightlyless ambiguity in the matching process.

Further, the addition of more sensors (for example, a charge-coupleddevice (CCD)) has the disadvantages of higher cost and larger size(along with manufacturing complications) relative to a single-sensorsystem. Moreover, multiple-sensor arrangements pose alignment challengesand robustness challenges; the multiple sensors are also differentlyaffected by temperature, vibration, and other environmental effects andas such are more prone to calibration errors.

For the foregoing reasons, there is a need for a quantitative 3-Dimaging system which either alleviates or eliminates the matchingproblem. The system should be viable in a single-lens, single-sensorarrangement for simplicity and compactness and also should be easilyexpandable to a multiple-lens, multiple-sensor arrangement if sodesired.

SUMMARY OF THE INVENTION

It is an objective of the present invention to provide a method forusing a single-lens device for three-dimensional imaging of a surface ofan object. The method comprises acts of: providing an imaging lens, anaperture configured to generate distinguishable images, a sensor, and aprocessor; transmitting light containing object information through theimaging lens and the aperture such that the transmitted light producesdistinguishable images on the sensor; generating a plurality ofdistinguishable images, each of the plurality of distinguishable imagesincluding point information of the surface of the object; calculatingdistance information from the point information between thedistinguishable images; and using the processor and distance informationto generate a representation of the object.

In another aspect of the present invention, the act of providing theimaging lens, the imaging lens includes an aperture plane, and furthercomprises an act of placing the aperture in the aperture plane of theimaging lens. In another aspect of the present invention, the act ofproviding an aperture configured to generate distinguishable images, theaperture is configured to generate distinguishable images by selectivelytransmitting light containing object information according to lightcharacteristics.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the aperture isconfigured to generate distinguishable images that change as a functionof time such that the distinguishable images are allowed to pass throughthe aperture as a function of time.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the apertureincludes a series of different shapes such that the transmitted lightthrough the aperture produces distinguishable, shape-based images.

In another aspect of the present invention, further comprising an act ofplacing the aperture in a location selected from a group consisting ofbeing in front of the imaging lens and being behind the imaging lens.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the aperture isconfigured to generate distinguishable images by selectivelytransmitting light containing object information according to lightcharacteristics.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the aperture isconfigured to generate distinguishable images that change as a functionof time such that the distinguishable images are allowed to pass throughthe aperture as a function of time.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the apertureincludes a series of different shapes such that the transmitted lightthrough the aperture produces distinguishable, shape-based images.

In another aspect of the present invention, in the act of providing theimaging lens, the imaging lens includes an aperture plane, and furthercomprising an act of providing a relay lens system such that theaperture is positioned at the aperture plane.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the aperture isconfigured to generate distinguishable images by selectivelytransmitting light containing object information according to lightcharacteristics.

In another aspect of the present invention, in the act of providing anaperture configured to generate distinguishable images, the aperture isconfigured to generate distinguishable images that change as a functionof time such that the distinguishable images are allowed to pass throughthe aperture as a function of time.

In yet another aspect of the present invention, in the act of providingan aperture configured to generate distinguishable images, the apertureincludes a series of different shapes such that the transmitted lightthrough the aperture produces distinguishable, shape-based images.

In another aspect of the present invention, a method for constructing arepresentation of a surface of a target object by using an addressablepattern is provided. The method comprising acts of: acquiring aplurality of images of an addressable pattern from different viewpoints,each of the plurality of images comprising addressable patterninformation and at least one point representing at least one aspect of atarget object; processing the plurality of images to assign an addressto each point in each of the plurality of images; developing anaddressed list of points using the address of each point in each of theplurality of images; returning the addressed list of points to animaging system processor for conversion into a three-dimensionalmeasurement; and causing the imaging system processor to generate arepresentation of a surface of a target object based on thethree-dimensional measurement.

In another aspect of the present invention, the method further comprisesan act of projecting the addressable pattern image onto the surface ofthe target object to generate a plurality of images of an addressablepattern.

In another aspect of the present invention, in the act of projecting theaddressable pattern image onto the surface of the target object, theprojected addressable pattern image is a pattern selected from a groupconsisting of a color sequence pattern and a shape sequence pattern.

In another aspect of the present invention, in the act of projecting theaddressable pattern image onto the surface of the target object, theprojected addressable pattern image is a position sequence pattern.

In another aspect of the present invention, in the act of physicallyplacing an addressable pattern image onto the surface of the targetobject.

In another aspect of the present invention, in the act of projecting theaddressable pattern image onto the surface of the target object, theprojected addressable pattern image is a pattern selected from a groupconsisting of a color sequence pattern and a shape sequence pattern.

In another aspect of the present invention, in the act of projecting theaddressable pattern image onto the surface of the target object, theprojected addressable pattern image is a position sequence pattern.

In another aspect of the present invention, in the act of providing atarget object such that the addressable pattern is inherent in thetarget object being imaged.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects, features and advantages of the present invention will beapparent from the following detailed descriptions of the disclosedaspects of the invention in conjunction with reference to the followingdrawings, where:

FIG. 1A is an illustration of a band-pass filter system that includes asensor;

FIG. 1B is an illustration of a defocused, multiple, color-coded imageacquisition of real points as received by the sensor of FIG. 1A;

FIG. 1C is an enhanced-view illustration of the framed area of FIG. 1B,demonstrating the matching procedure for a multi-wavelength addressablepattern in the form of a red dot and its corresponding green dot;

FIG. 1D is a chart illustrating the relationship of focal length (L) toZ-distance of matches and “ghost” particles with respect to FIG. 1C;

FIG. 2 is an illustration of a polarized filter imaging system;

FIG. 3 is an illustration of an aperture system for imaging points;

FIG. 4A is an illustration of a synched, single-aperture system with asinglehole mask shown in a first position A;

FIG. 4B is an illustration of a synched single-aperture system with asingle hole mask shown in a second position B;

FIG. 5A is an illustration of a single-aperture system having multiplef-stops;

FIG. 5B is an image acquired from the sensor of a single-aperture,multiple f-stop system;

FIG. 5C is an enhanced-view illustration of the framed area of FIG. 5B;

FIG. 5D is an illustration of a chart of matched points as determined bya processor;

FIG. 5E is an illustration of a vibrating, single-aperture system;

FIG. 6A is an illustration of an electronically masked imaging systemwith a first, multi-window electronic aperture open;

FIG. 6B is an illustration of an electronically masked imaging systemwith a second, multi-window electronic aperture open;

FIG. 7A is an illustration of an addressable template pattern suitablefor projection onto a surface of an object of interest;

FIG. 7B is an illustration of an acquired image taken of a target objectusing an addressable template;

FIG. 7C is an illustration of an acquired image and partial grid;

FIG. 7D is an illustration of the reconstructed illustration of thecenter sample of FIG. 7C;

FIG. 8A is a non-laser pattern projector and imaging system;

FIG. 8B is a two prism off-set and two-sensor system;

FIG. 8C is a one silvered offset prism and two-sensor system;

FIG. 8D is a three CCD-sensor assembly system;

FIG. 8E is a narrow-band mirror sensor assembly system;

FIG. 9 is a laser pattern projector and imaging system;

FIG. 10 is a flow chart depicting the steps of acquiring and processingimages in order to develop a representation of the surface of an object;

FIG. 11 is a flow chart depicting the incorporation of an addressablepattern to an imaging system in order to aide in image reconstruction;

DETAILED DESCRIPTION

The present invention satisfies the long felt need for an inexpensiveand precise way of three-dimensional imaging (e.g., mapping). Aspects ofthe invention are applicable to surface and volume inspection ofmanufactured parts, comparing actual products versus the originaldesign, scanning of 3-D objects, evaluation of body parts (hernias,arteries, pre- and post-plastic surgery, etc.), surface roughnessevaluation, and real-time feedback of surface deformation. In thefollowing detailed description, numerous specific details are set forthin order to provide a more thorough understanding of the presentinvention. However, it will be apparent to one skilled in the art thatthe present invention may be practiced without necessarily being limitedto these specific details. In other instances, well-known structures anddevices are shown in block diagram form, rather than in detail, in orderto avoid obscuring the present invention.

The reader's attention is directed to all papers and documents which arefiled concurrently with this specification and which are open to publicinspection with this specification, and the contents of all such papersand documents are incorporated herein by reference. All the featuresdisclosed in this specification, (including any accompanying claims,abstract, and drawings) may be replaced by alternative features servingthe same, equivalent or similar purpose, unless expressly statedotherwise. Thus, unless expressly stated otherwise, each featuredisclosed is one example only of a generic series of equivalent orsimilar features.

Furthermore, any element in a claim that does not explicitly state“means for” performing a specified function, or “step for” performing aspecific function, is not to be interpreted as a “means” or “step”clause as specified in 35 U.S.C. Section 112, Paragraph 6. Inparticular, the use of “step of” or “act of” in the claims herein is notintended to invoke the provisions of 35 U.S.C. 112, Paragraph 6.

First, an introduction to the present invention is provided to give anunderstanding of the general aspects. Second, defocusing methods basedon light properties and mask shape are discussed with respect to featuremapping. Third, aspects of single aperture systems are discussed withrespect to feature mapping. Fourth, examples of pattern matching areprovided. Fifth, imaging methods according to the present invention areprovided. Finally, a discussion of image matching is provided.

(1.0) Introduction

Blur from defocus can be used to measure the distance between a pointand the focal plane of a lens. Proposed are added dimensions in terms ofoptical and illumination techniques to the single-lens multiple-aperturearrangement that overcome the shortcomings of the original defocusingconcept. The following aspects allow for robust measurement of an objectsurface with a single-lens, single-sensor, and multiple-aperture device.

Optical modifications to the multiple-aperture arrangement physicallymask and convey filtered information to the sensor in such a way thateach aperture produces a separable image for reconstructing an objectsurface. In order to produce a separable image, the aperture mask may bemodified by altering the shape of the aperture, by coding thetransmittance of the aperture, or providing a single-slit mask whosehole moves about the aperture plane during or between exposures. Each ofthe aperture masks provides additional information which aides inrepresenting the desired features of an object.

A single-lens, single-sensor, multiple aperture device may be furtheraugmented to obtain additional information from the object by usingregistered information. Registered information may be obtained from thedistinguishing characteristics of the object, from information projectedonto the surface of the object, or from information or markers placeddirectly onto the object.

For large objects which cannot be captured with a single exposure, theaforementioned aspects may provide information which may be used to fitmultiple exposures together in order to recreate surface features of adesired object. Alternatively, multiple images can be scanned in forboth large and small objects in order to produce a high resolutionrepresentation of the object or object feature. The matching concept isequally applicable to stereo vision systems.

Aspects of the present invention will now be described more fullyhereinafter with reference to the accompanying drawings, in whichpreferred embodiments of the invention are shown. This invention may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein. Further, the dimensions of layersand other elements shown in the accompanying drawings may be exaggeratedto more clearly show the details. The present invention should not beconstrued as being limited to the dimensional relations shown in thedrawings, nor should the individual elements shown in the drawings beconstrued to be limited to the dimensions shown.

(2.0) Light Property and Shape-Based Systems

A masked aperture generates a distinguishable image as light from anilluminated object is passed through a lens, through a masked aperture,and onto a sensor suitable for receiving the information from the maskedaperture. The masked aperture passes coded and defocused information ofthe object onto a suitable sensor. The defocused information provides ameasurement of a point on an object relative to the focal plane. Thecoded information from the masked aperture provides the informationrequired in order to separate overlapping images and match correspondingpoints detected by the sensor.

When two or more masked apertures are used, each mask is ideallydifferent from the other such that the intensity versus wavelengthproperties and/or morphology of detected shapes from the maskedaperture(s) is easily distinguishable on the sensor. A variety offiltering apertures may be used in order to selectively filter lightaccording to its properties onto a light sensor such that the imagesfrom each aperture are distinguishable. Further, when the shapes of twoor more apertures are distinguishable, each aperture image detected bythe sensor is also distinguishable. Therefore, non-limiting examples ofsuitable aperture masks and filters include wavelength band-passfilters, light polarization filters, and differentially shaped masks.

(2.1) Color Coded Filters

Referring to FIG. 1A, a band-pass filter system 100 is shown. Theband-pass filter system 100 includes a lens 102, a mask 104 having a redaperture 106 and a green aperture 108, and a sensor 110. Although shownas a red and green aperture 106 and 108 respectively, any number andcombination of color filtered aperture may be used in combination withan appropriate sensor 110. Thus, while the apertures are referred tospecifically as the color red and green apertures 106 and 108, they arenot intended to be limited thereto and could alternatively be referredto as a first aperture, a second aperture, and so forth.

The band-pass filter system 100 produces a representation of theilluminated object 112 when placed in front of the focal plane 114.Scattered light 116 is reflected form the surface of the illuminatedobject 112 and through the lens 102. Once through the lens 102, thescattered light 116 selectively passes through either the red aperture106 or the green aperture 108, or is reflected off of or absorbed by themask 104. The transmitted red light 118 from the red aperture 106produces a transmitted green light 120 from the green aperture 108 isthen recorded on the sensor 110 positioned in front of the focal imagepoint 122.

Referring to FIG. 1B, a defocused, multiple color-coded imageacquisition 124 of real points are shown as received by the sensor 110of FIG. 1A. Each color-coded acquisition 124 corresponds with amulti-wavelength addressable pattern created by the respective aperture106 and 108. As shown in FIG. 1B, each real point on the object isrepresented with multi-wavelength addressable pattern red dots 126 andgreen dots 128. As can be appreciated by one skilled in the art, the redand green dots 126 and 128 are a result of the red and green apertures,respectively; however, the invention is not limited thereto as the colorof the dots would vary according to the color and combination of theapertures. Corresponding red dots 126 and green dots 128 are shownlinked together with a correspondence line 130. The correspondence lines130 are not visible; however, they are a useful illustration toolhighlighting the difficulty of matching points in color-coded imageacquisitions 124. Only the dots connected by correspondence lines 130actually correspond together. Without the mask 104, there would not beenough information to link corresponding points.

Referring to FIG. 1C, an enhanced-view of the framed area 132 of FIG. 1Billustrates the matching procedure between a corresponding red dot 126and green dot 128. When the multiple color-coded image acquisition 124has been developed, a processor then begins a search for all of thecolor coded dots within the image. Alternatively, the search may beconducted from raw data (i.e., an image 124 need not be produced).Instead, the sensor 110 is coupled to a processor which receives thesensor information directly. In either case, once all dots have beendetected, the matching process begins with an assumption of the relativeposition of the illuminated point 112 with respect to the focal plane114. The relative position of the illuminated point 112 with respect tothe focal plane 114 may be known a priori, entered by a user, determinedby software, or determined by sensors. Here, it is postulated that theilluminated point 112 of the object is in front of the focal plane 114.Therefore, the matching begins with the instruction command, forexample: “Any green dot 128, 136, 138, and 140 to the right of a red dot126, 142, 144, and 146 on a line corresponding to a line connecting thetwo apertures (within a tolerance) is a match.” The first red dot 126 isdetected, and then matched to the first green dot 128 within tolerance134 of the red dot 126 according to the instruction command.Supplemental searches conducted for green dots 136, 138, and 140 withinthe tolerance 134 of the red dot 126 produces a total of three “ghost”matches (green dots 136, 138, and 140, respectively).

Referring to FIG. 1D, the relationship of focal length (L) to Z-distanceof matches and “ghost” particles with respect to FIG. 1C is shown. Thematching of the red dot 126 to all of the green dots 128, 142, 144, and146 results in one match 148 and three ghosts 150, 152, and 154. Thematch between the red dot 126 and the green dot 128 is used to calculatethe Z-to-L relationship of the first matched point 148. The mismatchbetween the red dot 126 and the green dots 136, 138, and 140 providesthe first three ghosts 150, 152, and 154, respectively.

With respect to the second red dot 142, one match 156 and two ghosts 158and 160 are produced. The match between the second red dot 142 and thecorresponding green dot 136 is used to calculate the Z-to-L relationshipof the second matched point 156. The mismatch between the red dot 142and green dots 138 and 140 is represented by the two ghosts 158 and 160respectively.

With respect to the third red dot 144, one match 162 and two ghosts 158and 160 are produced. The match between the third red dot 144 and thecorresponding green dot 138 is used to calculate the Z-to-L relationshipof the 10 third matched point 162. The single mismatch between the reddot 144 and green dot 140 is represented by the ghost 164.

Finally, with respect to the fourth red dot 146, one match 162 but noghosts are generated. The match between the fourth red dot 146 and thecorresponding green dot 140 is used to calculate the Z-to-L relationshipof the fourth and final matched point 166. Since there are no othergreen dots to the right of the red dot 146 other than the matching greendot 140, no additional mismatches exist for the framed area 132 of FIG.1C.

Determining the relationship of focal length (L) to Z-distance betweenmatches and “ghost” particles is greatly enhanced by differentiallycoded points, such as those shown 126 and 128 in FIG. 1B. In anon-separable case, one in which there is no color information providedby an aperture mask 104, there are many more ghosts because, withouthaving a differentiator like color, each “red dot” of FIG. 1A can bematched with any other “red dot” producing many more ghosts. Further, noassumptions can be made that any given dot by itself is not in fact twodots on top of the other, adding even more ghosts at the focal plane.

(2.2) Polarized Filters

Coded information may be provided to a sensor in any number of ways. Asa non-limiting example, FIG. 2 illustrates a polarized filter imagingsystem 200. The polarized filter imaging system 200 includes a lens 202,a mask 204 having a horizontal polarizing aperture 206 and a verticalpolarizing aperture 208, and a sensor 210 capable of distinguishingbetween polarizations. Although shown as a combination of horizontallyand vertically polarized apertures 206 and 208 respectively, any numberand combination of at least nearly orthogonal pairs of orientations maybe used.

The polarized filter imaging system 200 produces a representation of theilluminated object 212 when placed in front of the focal plane 214.Scattered light 216 is reflected form the surface of the illuminatedobject 212 and through the lens 202. Once through the lens 202, thescattered light 216 selectively passes through either the horizontalpolarizing aperture 206 or the vertical polarizing aperture 208, or isreflected off of the mask 204. The transmitted horizontally polarizedlight 218 from the horizontal polarizing aperture 206 and thetransmitted vertically polarized light 220 from the vertical polarizingaperture 208 is then recorded on the sensor 210 positioned in front ofthe focal image point 222.

By differentially coding the horizontal polarizing aperture 206 and avertical polarizing aperture 208, distinguishable dots, similar to thoseshown in FIG. 1B, are obtained. However, the coded information obtainedfrom the present polarized aspect provides polarization markers insteadof color coded dots.

Selectively transmitting light (as is the case with a band-pass filtersystem 100) or exploiting properties of light (as is the case with apolarized filter imaging system 200) are effective means of codinginformation received by a sensor. Ultimately, the coded informationdetected by the sensor eases the task of matching described with respectto FIG. 1C and FIG. 1D.

(2.3) Physical Mask Shape

Referring to FIG. 3, a differentially-shaped aperture system 300 forimaging points small enough to be considered nearly point sources, isshown. The differentially-shaped aperture system 300 includes a lens302, a mask 304 having a circular-shaped aperture 306 and asquare-shaped aperture 308, and a sensor 310. Although shown as acircular-shaped aperture 306 and a square shaped aperture 308, anynumber and combination of different shape-filtered apertures may beused. Non-limiting examples of suitable shapes include convexpolyhedrons, concave polyhedrons, circular shapes, polyforms, andcombinations thereof.

The differentially-shaped aperture system 300 produces tworepresentations 314 and 316 of the illuminated object 312 per exposure.Each shape 314 and 316 detected by the sensor 310 corresponds to theshape of the respective 15 aperture 306 and 308, respectively. Asscattered light 320 is reflected off the surface of the illuminatedobject 312 and through the lens 302, it will either pass through thecircular-shaped aperture 306, the square-shaped aperture 308, or bereflected by the mask 304 and beyond the sensor focal plane 318. Thetransmitted light 322 which passes through the circular-shaped aperture306 produces a circular pattern 314 on the sensor 310. Similarly, thetransmitted light 324 which passes through the square-shaped aperture308 produces a square pattern 316 on the sensor 310. After multipleacquisitions, the numerous circular patterns 314 and square patterns 316are detected and then matched by a processor 326 based upon a matchingrule. Both the matches and ghosts may then be plotted on a Z-to-L plot,such as the one depicted in FIG. 1D. Alternatively, a plot demonstratingthe matches without ghost images may also be generated.

(3.0) Single Aperture System

Referring to FIG. 4A and FIG. 4B, a synced single-aperture system 400includes a lens 402, a single-hole mask 404, a moving aperture 406, asensor 408, and a processor 410 in communication with the sensor 408, isshown. Additionally, the single-hole mask 404 is shown in a firstposition A and second position B, respectively. An illuminated object412 may be reconstructed by selectively allowing reflected rays 414 topass through the lens 402 and the long and narrow slit-aperture 406 ofthe single-hole mask 404. The position of the single-hole mask 404,whose aperture 406 moves about the aperture plane between exposures, isrecorded by the processor 410. As shown in FIG. 4A, the moving aperture406 transmits light 416 and produces a first point 414 detected by thesensor 408. The first position information of the moving aperture 406during the first exposure is recorded by the processor 410. For thesecond exposure, the moving aperture 406 is moved to the second positionB. As shown in FIG. 4B, the moving aperture 406 transmits light 418 andproduces a second point 420 detected by the sensor 408. The secondposition information of the moving aperture 406 during the secondexposure is recorded by the processor 410. The first point 414 and firstposition information and second point 420 and second positioninformation are then used to match the first point 414 from the firstexposure with those of the second point 420. Alternatively, the color ofthe reflected rays 414 may be altered between the first exposure andsecond exposure in order to provide additional information which may beused to aide in the matching process.

(3.1) Single Slit-Aperture System

Referring to FIG. 5A, a single-aperture system 500 having multiplef-stops is illustrated. The differentially-shaped aperture system 500includes a lens 502, a mask 504 having a long and narrow slit-aperture506, a sensor 508, and a processor 510 in communication with the sensor508. Although shown as a roughly oblong-shaped aperture 506, in general,any aperture which is significantly longer in length than in width maybe used regardless of shape. Also, the slit aperture 506 could be madeto be asymmetric in shape such that the front and back of the focalplane can be distinguished.

An illuminated object 512 may be reconstructed by selectively allowingreflected rays 514 to pass through the lens and the long and narrow slitaperture 506 of the mask 504. Notably, the single-aperture system 500uses a long, narrow, slit-aperture 506, instead of a standard circularaperture. Effectively the slit aperture 506 has a different f-number intwo directions. The long length of the slit aperture 506 produces a lowf-number which generates a large variance disc 516 on the sensor 508.Conversely, the narrow width of the slit aperture 502 produces a highf-number, generating a minimum variance, such that the image of a pointsource is represented by lines 518 rather than discs 516. The intensitycan now be thought of as varying inversely with length rather than area,so the dynamic range required on the sensor is much decreased relativeto a pure-blur system. Further, the size of the produced images 516 and518 only increase in one direction, minimizing the chance for overlap.

Referring to FIG. 5B, an image 520 acquired from the sensor 508 of asingle-aperture multiple f-stop system 500 is shown. Within the frame522 of the image 520, multiple plots 524, 526, 528, and 530 withdifferent Z-coordinates are shown. Although shown as an image 520, theinformation depicted may also be conditioned and sent via a signal to aprocessor 510 for processing.

Referring to FIG. 5C, the framed area 522 of the acquired image of FIG.5B is processed in order to find the multiple f-stop streaks 524, 526,528, and 530 corresponding with the aperture movement. Once all of themultiple f-stop streaks 524, 526, 528, and 530 have been found, a ruleis applied to determine the Z-to-L relationship. Notably, no matching isrequired.

By assuming all of the points were in front of the focal plane “L,” themultiple f-stop streaks 524, 526, 528, and 530 are used to calculate theZ-to-L relationship. An example of matched points 532, 534, 536, and 538determined by a processor 510 are shown in FIG. 5D. In general, theprocessor 510 connected with the sensor 508 may be used to collect theraw data obtained from the sensor. The processor 510 then may use theZ-to-L relationships in order to calculate the depth information of eachdetected f-stop streaks 524, 526, 528, and 530. The processor 510 maythen be used to generate a representation of the object from the depthinformation of each illuminated point 512. In another aspect, theprocessor 510 may also include memory. The memory may be used to storecalibration information of previously sampled points at known distances.The calibration information may be stored as a look-up table in theimage-acquisition system for fast inline processing. Alternatively, thecalibration information may be stored remotely and accessed by theprocessor.

The results depicted in FIG. 5B, FIG. 5C, and FIG. 5D may also beobtained by using a vibrating, single-aperture system 540 such as theone illustrated in FIG. 5E. The vibrating, single-aperture system 540includes a lens 502, a mask 504 having a single moving aperture 506, anda sensor 508.

An illuminated object 512 may be reconstructed by selectively allowingreflected rays 514 to pass through the lens and the long and narrow slitaperture 506 of the mask 504. Notably, the single-aperture system 500uses a moving aperture 506, effectively simulating the effect of havinga different f-number in two directions. As the moving aperture 506controllably oscillates right to left in the direction of A and B (or inany other suitable direction), the net displacement of the movingaperture 506 from A to B produces a low f-number. The low f-number ofthis lateral movement from A to B generates a large variance disc 516 onthe sensor 508. Further, as the moving aperture 506 moves from A to B,there is no net change to the vertical diameter of the moving aperture506. The constant height of the moving aperture 506 therefore produces ahigh f-number, generating a minimum variance, such that the image of apoint source is represented by lines 518 rather than discs 516.

The intensity is dependent upon the amount of time the aperture 506spends at a particular axial position, thus the images would look morelike bright ends connected by dimmer straight lines. Further, the sizeof the produced images 516 and 518 only increase in one direction,minimizing the chance for overlap.

In one aspect, the invention can be thought of as a two-aperture systemwith the ambiguity of matching removed by simply connecting the twoaperture images physically on the imager. When imaging large objectsthrough the aperture (not point sources), three images are visible. Thecenter image is the image of the object, and the outer two images areformed as a result of diffraction and lens effects. As the scale of theobject decreases, it approaches a point source, and at the limitingcase, the image of the point source object has the same shape as theaperture.

(3.2) Electronic Masked Aperture

Referring to FIG. 6A and FIG. 6B, an electronically masked imagingsystem 600 including a lens 602, an aperture plate 604, a multi-windowelectronic aperture 606, a sensor 608, and a processor 610 incommunication with the sensor 608 and aperture plate 604, is shown.Non-limiting examples of suitable aperture plates 604 include a liquidcrystal display (LCD) which may be fully synchronized with theacquisition sensor 608. In one aspect, the sensitivity may be controlledby varying the “off-axisness” of the apertures. An illuminated object614 may be reconstructed by selectively allowing reflected rays 616 topass through the lens 602 and one of the many windows of themulti-window electronic aperture 606.

As shown in FIG. 6A, a first window 612 of the multi-window electronicaperture 606 transmits light 618 and produces a first point 620 detectedby the sensor 608. During the first exposure, the first open window 612position information is recorded by the processor 610.

To obtain a second exposure, a second window of the multi-windowelectronic aperture 606 is opened. As shown in FIG. 6B, a second window622 of the multi-window electronic aperture 606 transmits light 624 andproduces a second point 626 detected by the sensor 608. During thesecond exposure, the second open window 622 position information isrecorded by the processor 610. The first point 620 and first open window612 position information and second point 626 and second position openwindow 622 position information are then used to match the first point620 from the first exposure with the information of the second point626.

(4.0) Pattern Matching

If the object of interest is a surface whose shape is to be matched, apattern of markers may be projected on the surface and the points in thedetected image may be sampled by measuring the relative position of theprojected markers. The sensor's allowable upper density limit of imageddots is the imaging system's limit. Once the points are identified ineach view, there is only the question of whether the same point existsin both views. In another aspect, if the volume to be mapped contains acluster of asymmetrical cells in a volume, then the shape andorientation of the cells can be used as an additional constraint in theinter-view matching, thus reducing the chance that a mismatch can occur.This aspect is referred to as “feature matching.”

Referring to FIG. 7A, an illustration of an addressable template pattern700 suitable for projection onto the surface of the object of interestis shown. The addressable template pattern 700 is projected orphysically placed on the target surface and then captured by an imagingsystem at varying distances (Z) from the object. In one aspect, theaddressable template pattern 700 is in the form of a grid pattern with adistinguishable center point 702. Referring to FIG. 78, an illustrationof an acquired image 704 taken of a target object using an addressabletemplate is shown. As illustrated, some dots 706, 708, 710, and 712 aremissing in the acquired image 704.

Referring to FIG. 7C, the acquired image 704 with a partial grid 714 isshown. If the addressable pattern 702 is ordered, a grid pattern 714with a distinguishable origin can employ a method such as “structuredpattern matching” to reduce the number of required viewpoints, or imageacquisitions, to two. The addressing algorithm, such as one stored on acomputer readable medium or executed by a processor, processes eachaperture's image to “find the relative address” of each dot in theaddressable pattern 704 according to the template pattern. Anon-limiting example of a suitable addressable template pattern 700 isillustrated in FIG. 7A. The addressing algorithm has some tolerance toallow for deformation of the addressable pattern 704 (See FIG. 7B andFIG. 7C). The deformation of the addressable pattern 704 is noticeablewhen contrasted with the original addressable template pattern 700 (SeeFIG. 7A). Further, the addressing algorithm can also account for missingentities 706, 708, 710, and 712 in the acquired image 704. Missinginformation is considered missing when a point on the addressabletemplate pattern 700 does not appear in the addressable pattern 704.

A reconstructed illustration of the center sample 716 of FIG. 7C isillustrated in FIG. 7D. The points are reconstructed by calculating theZ for each pair of dots with the same address. Any pair with a missingdot is not reconstructed.

(4.1) Pattern Projector (Non-Laser)

Referring to FIG. 8A, a non-laser pattern projector 800 and imagingsystem 802 are shown. The non-laser pattern projector 800 comprises alens 804 identical to the imaging lens 806 of the imaging system 802.The lens 804 of the non-laser pattern projector 800 is placed at anequivalent distance from the beam splitter 808 as the lens 806 of theimaging system 802. This causes the principal rays 810 of the projectedpoints 812 to coincide with the principal rays 814 detected by thesensor 816 of the imaging system 802. Thus the projected pattern 818will look as though it does not move in the detected image, even whenthe distance between the projected point 812 and the focal plane 820 ofthe imaging lens 806 changes. This makes identifying an addressablepattern 818 much easier, even if some points (e.g., dots) are missing.

The prerequisite is that the images from each viewpoint are physicallyseparate—this is naturally true in multiple-sensor systems such asphotogrammetry, but requires special care with systems like thedefocusing concept (multiple apertures on a single lens imaging onto asingle sensor).

The projected pattern 818 is produced by passing light 822 through apattern stencil 824 and projector lens system 826 with a lens 804substantially identical to the imaging lens 806.

For single-lens systems, the aperture images must be separate. This canbe accomplished with prisms (See FIG. 8B and FIG. 8C) or fiber opticbundles so that each aperture projects onto a separate sensor, or with aphysically masked aperture (See FIG. 8D and FIG. 8E) if the sensor is acolor sensor.

Referring to FIG. 8B, a two prism off-set and two-sensor system 828 isshown. The system 828 comprises a first prism 830, second prism 832, anda first sensor 834 and second sensor 836 behind a mask and two-slitaperture 838. The first prism 830 and second prism 832 offset theincoming light 840 and 842 from the two-slit aperture 838 such thatlight transmitted through the first prism 830 and second prism 832 maybe detected by separate sensors 834 and 836. Such a configuration may beused when the two-slit aperture 838 is used to code information based onthe inherent properties of light or the light must be separated as isthe case when addressable pattern techniques are employed. Non-limitingexamples of suitable inherent properties include but are not limited tothe frequency, frequencies, or polarization of coded transmitted lightdetected images.

Referring to FIG. 8C, a one silvered offset prism and two-sensor system844 is shown. The system 844 comprises a silvered prism 846, a firstsensor 848 and second sensor 850 behind a mask and two-slit aperture852. The silvered prism 846 offsets the first bundle of incoming light854 from the two-slit aperture 852 such that light transmitted throughthe silvered prism 846 may be detected by the first sensor 848.Alternatively, light 856 which has passed through two-slit aperture 852may also be detected separately on the second sensor 850.

Referring to FIG. 8D a three CCD-sensor assembly system 858 is shown.The system 858 comprises a three CCD-sensor 860 behind a mask andtwo-slit aperture 862. The CCD-sensor 860 includes a blue sensor 862, agreen sensor 864, and a red sensor 866. The system of prisms 868 offsetsthe first bundle of incoming light 870 from the two-slit aperture 856such that light transmitted through the prism 868 may be detected by thered sensor 866. Alternatively, light 872 which has passed through thetwo-slit aperture 852 may also be detected separately on the greensensor 864.

FIG. 8E is a narrow-band mirror sensor assembly system 874 is shown. Thesystem 874 comprises a narrow-band mirror 876, located behind a mask andtwo-slit aperture 878, and a first sensor 880 and second sensor 882. Thesystem narrow-band mirror 876 offsets the first bundle of incoming light884 from the two-slit aperture 878 such that light transmitted throughthe narrow-band mirror 876 may be detected by the first sensor 880.Alternatively, light 886 which has passed through the two-slit aperture878 may be detected separately on the second sensor 882.

(4.2) Pattern Projector (Laser)

Any lens can be represented by two “principal planes.” The location ofthe planes is only a function of the lens, and all principal rays (whichdefine the image centerline for a point) behave as if they entered thefirst principal plane and exited the second principal plane at the axis.

By measuring the location of the front principal plane and the field ofview, a diffraction grating with the desired pattern can be made andpositioned such that the beams from the laser projector coincide withthe principal rays of the imaged dots. Thus, the projected pattern willlook as though it does not move in the image even when the distancebetween the projected dot and the focal plane of the imaging lenschanges. This makes searching for the addressable pattern much easiereven if some dots are not imaged.

A complex ray trace through a compound lens (where the ray kinks atevery air/glass interface) can be mathematically represented as twoplanes at which the rays kink. Thus, the left image shows the “real” raytrace, and the right image shows the mathematical representation of suchlens. The planes are found by taking any chief (also called principal)ray coming into the first glass interface and leaving the last glassinterface and extending them to intersect the axis of the lens. Theintersection marks the location of the planes.

Thus, one would first do a calibration (by imaging a grid at severalZ-distances) and then do a least-squares type fit to find out wherethose two planes are, and what the field of view angle is. Then, thediffraction grating can be customized to match the field of view angle,and put at the same distance from the beam-splitter as the firstprincipal plane. Therefore, the laser beams will follow exactly the pathof the principal rays. In operation, an object, as it gets closer to alens, gets bigger in the image. This means that the edges of the objectmove laterally on the image. The same would be true of any patternprojected in front of the camera onto a surface. By making the raysmatch exactly as in the aspect depicted in FIG. 9, none of the pointsever move laterally, regardless of their Z-position.

Now, if a two-hole aperture mask is added, the corresponding dots stillmove apart from each other (the dots (images) are formed by themarginal, or outer, rays). However, since the chief ray is not movinglaterally, the centroid of the corresponding “match shape” will not movelaterally. Conceivably, once the distinguishable dot of the addressablepattern is located, the centroid of that match can be found. Knowingthat the pattern is never expanding laterally, it is known where thecentroid of every other point on the pattern should be, which should aidin “addressing” the points. This is different than the traditionaladdressable pattern search, where the points are all moving relative toeach other, so that if there's too much of a surface Z-change, thepattern may not be reconstructable.

Referring to FIG. 9, a laser pattern projector system 900 and imagingsystem 902 are shown. The laser pattern projector system 900 comprises alaser projector 904 and a filtering mask 906. The filtering mask 906selectively passes light from the projector 904 onto the fifty percentbeam splitter 908. The laser projector 904 and a filtering mask 906 arein-line with the beam splitter 908 which causes the principal rays 910of the projected points 912 to coincide with the principal rays 914detected by the sensor 916 of the imaging system 902. Thus the projectedpattern 918 will look as though it does not move in the detected image,even when the distance between the projected point 912 and the focalplane 920 of the imaging lens 906 changes. This makes identifying anaddressable pattern 918 much easier, even if some points (e.g., dots)are missing.

(5.0) Imaging Methods

Referring to FIG. 10, a flow chart depicting the steps of acquiring andprocessing images to develop a two dimensional or three dimensionalrepresentation of the surface of an object is shown. Any single-lensdevice may be built or modified to include an imaging lens, an apertureconfigured to generate distinguishable images, a sensor, and aprocessor.

The imaging process begins by illuminating the surface of the object1000. The surface may illuminated by the imaging system or a suitableexternal lighting source. Light is reflected off of the surface of theobject and transmitted through the aperture 1010. The aperture may beplaced in the plane of the imaging lens, in front of the imaging lens,behind the imaging lens, may be applied at an aperture plane of theimaging lens when accessible, or made accessible via a relay lenssystem.

As the light travels past the aperture, the aperture may be used in anumber of ways to code information received by the sensor. Non-limitingexamples of suitable methods by which light may be coded in order toproduce distinguishable images 1020 on the sensor include but are notlimited to: filtering transmitted light according to lightcharacteristics (such as filtering by wavelength or polarization),transmitting light as a function of time such that the distinguishableimages are allowed to pass through the aperture as a function of time;or physically altering the shape of the aperture to comprise a series ofdifferent shapes from which transmitted light through the apertureproduces distinguishable shape-based images.

A step may be also be implemented which aids the system in determiningwhether or not to acquire additional images 1030. The step may furtherbe augmented to weigh the suitability of an acquired image. For example,an image detected by a sensor which suffered from excess movement duringthe exposure may be discarded by the algorithm. In this case, the lastacquired image would be discarded and the process would be repeated withthe illumination of the object 1000. In another aspect, the receivedimage may be suitable for processing; however, more image acquisitionimages are needed 1030. In this case, a further decision can be added tofurther augment the algorithm. An example of which would be to add adecision to determine whether or not the viewpoint of the imaging systemshould be adjusted 1040. If the position of the imaging device or thedesired area of the object needs to be shifted, either the imagingsystem or the target object may be altered to adjust the viewpoint 1050.

Once all or at least some of the images have been acquired, therelationship amongst points, or point information, within each image isused to calculate or determine the relative or absolute distanceinformation for each point 1060. Once the distance information is known,the information may be fed to an algorithm which uses the distanceinformation to generate a representation (e.g., 3-D mapping) of theobject 1070.

(6.0) Image Matching

For large objects or applications which require multiple exposureacquisitions, image matching provides a method by which related imageacquisitions may be tied together to recreate an object surface.Although not required to recreate the target object, when the positionof the imaging system is known relative to the target object, imagematching offers the ability to recreate a target object with exactmeasurements. In general, image matching, also referred to as digitalquilting, is greatly aided by the use of an addressable pattern templateimage. In one aspect, the addressable pattern projector may bephysically tied to the acquisition device. In another aspect, theaddressable pattern projector may move independently of the device, butin such a way that the pattern visible by the device is stilladdressable.

An imaging device acquires an addressable-pattern template image at aninitial position. The addressable-pattern template image typically has afinite number of points in the X, Y, and Z-planes. The position of theimaging device is then adjusted and a second addressable-patterntemplate image is acquired at second position. Precautions may be takensuch that an adjusted position determined to exceed motion constraintsare ignored. The second position, or adjusted position, is related tothe initial imaging device position by a six-variable solid translationand rotation. Typically, the adjusted position is related to the initialposition by the fact that the image captured at the new positionoverlaps in part with the first template image and has a substantiallysimilar number of points.

In operation, at least one outer hull is generated by a processor or ismanually highlighted by the user. The outer hull encompasses all thepoints within the addressable-pattern template image andaddressable-pattern surface image. Although not always the case, thepoints outside the addressable-pattern template image outer hull may bedisregarded. A plurality of inner hulls of the points in theaddressable-pattern surface image is also generated. The inner hull is afunction of a maximum acceptable displacement between acquisitionswithin the intersection of the plurality of hulls, according to thesix-variable solid-body translation and rotation. The error may becalculated from the difference between a point on theaddressable-pattern surface image and the addressable-pattern templateimage.

When the hulls have been generated, the addressable-pattern informationis processed using a matching algorithm. The matching algorithm isconfigured to determine the distance between each point on theaddressable-pattern surface image and its corresponding point on theaddressable-pattern template image. Each of the matched points is thenmerged from the plurality of inner hulls according to their solid-bodytranslations and rotations to form a high-resolution data set.

When hundreds or possibly thousands of acquisitions have been matched,the point clouds are merged according to their solid-body translationsand rotations. An algorithm that uses the addressable patterninformation may also be adapted to determine whether or not enoughmatching points exist to recover the features of the target object. Whena high-resolution point cloud has been developed, the high-resolutionpoint cloud can be used to generate or output a high-resolution surface(nurbs, meshes, etc.) with or without interpolation via standardalgorithms or commercial packages, such as Geomagic Studio. GeomagicStudio is produced by Geomagic, located at 3200 East Hwy 54, Cape FearBuilding, Suite 300, Research Triangle Park, NC, 27709 U.S.A.

The fit is considered satisfactory if the total error is below somethreshold which is a function of the precision of the device. Once thisis done, second acquisition at the adjusted position becomes thetemplate and the next acquisition becomes the surface matched to it. Therobustness of addressable pattern information in the matching algorithmallows for the matching of small set to small set, without interpolatingthe surface shape until enough acquisitions are available.

Referring to FIG. 11, a flow chart depicting the use of an addressablepattern to aide in image reconstruction is shown. The use of anaddressable pattern during surface feature acquisition is one way ofbypassing the correspondence search employed in a separable-viewpointthree-dimensional imaging system.

The process begins with the acquisition of a number of images, each ofthe images containing (being illuminated with) an addressable pattern1100. Each image is typically taken from a different viewpoint, althoughthe addressable pattern is static with respect to the contours of theobject's surface. Each of the plurality of images comprising at least aportion of the addressable pattern information and at least one pointrepresenting at least one aspect of the target object. It will beappreciated that an object may include a variety of points on theobject. Each point may provide important information with respect to theeventual reconstruction of the object.

An address is assigned to each point in the image 1110. In general theaddressable pattern provides a sequence or series of plots on the objectwhich may be referenced to assist in the addressing step 1110.Importantly, the addressable pattern need not be symmetrical or containa regular sequence of markers or images. Non-limiting examples ofsuitable addressable pattern information may include a color sequencepattern, a pattern comprising differently shaped object, a positionsequence pattern, distinguishable object features or object landmarks,or any combination thereof. The addressable pattern image may be placedon the surface of the object in a variety of ways. Non-limiting examplesof suitable methods include: projecting the addressable pattern imageonto the surface of the object; physically placing an addressablepattern image onto the surface of the object; and using the featuresinherent to the object being imaged as a source.

A step may be also be implemented which aids the system in determiningwhether or not to acquire additional images 1120. The step may furtherbe augmented to weigh the suitability of an acquired image. For example,an image detected by a sensor which suffered from excess movement duringthe exposure may be discarded by the algorithm. In this case, the lastacquired image would be discarded and the process would be repeated withthe illumination of the object 1100. In another aspect, the receivedimage with an addressable pattern may be suitable for processing;however, more image acquisition images are needed to reconstruct theobject. In this instance, a further decision process can be added tofurther augment the algorithm. An example of which would be to add adecision to determine whether or not the viewpoint of the imaging systemshould be adjusted 1130. If the position of the imaging device or thedesired area of the object needs to be shifted, either the imagingsystem or the target object may be altered to adjust the viewpoint 1140.

Once all or at least some of the images have been acquired, therelationship amongst points, or point information, within each image isused to calculate or determine the relative or absolute distanceinformation for each point. Once the distance information is known, theinformation may be fed to an algorithm which uses the distanceinformation to generate a representation of the object 1160.

The drawings and the associated descriptions are provided to illustrateembodiments of the invention and not to limit the scope of theinvention. Reference in the specification to “one embodiment” or “anembodiment” is intended to indicate that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least an embodiment of the invention. The appearancesof the phrase “in one embodiment” or “an embodiment” in various placesin the specification are not necessarily all referring to the sameembodiment.

As used in this disclosure, except where the context requires otherwise,the term “comprise” and variations of the term, such as “comprising,”“comprises” and “comprised” are not intended to exclude other additives,components, integers or steps.

Also, it is noted that the embodiments are disclosed as a process thatis depicted as a flowchart, a flow diagram, a structure diagram, or ablock diagram. Although a flowchart may disclose various steps of theoperations as a sequential process, many of the operations can beperformed in parallel or concurrently. The steps shown are not intendedto be limiting nor are they intended to indicate that each step depictedis essential to the method, but instead are exemplary steps only.

In the foregoing specification, the invention has been described withreference to specific embodiments thereof. It will, however, be evidentthat various modifications and changes may be made thereto withoutdeparting from the broader spirit and scope of the invention. Thespecification and drawing are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It should be appreciatedthat the present invention should not be construed as limited by suchembodiments.

From the foregoing description, it will be apparent that the presentinvention has a number of advantages, some of which have been describedherein, and others of which are inherent in the embodiments of theinvention described or claimed herein. Also, it will be understood thatmodifications can be made to the device, apparatus and method describedherein without departing from the teachings of subject matter describedherein. As such, the invention is not to be limited to the describedembodiments except as required by the appended claims.

1. An image sensor assembly, comprising: an optical system, including animaging lens, an optical aperture in an optical train that includes saidimaging lens and an object being imaged, said aperture configured toconstrain an outer shape of passed information; an image sensor, also insaid optical train, and receiving image information that has passedthrough said imaging lens and said optical aperture, and producing anelectrical output indicative of said image information; a processorreceiving raw information directly from said image sensor, saidprocessor analyzing said raw information to recognize multipledistinguishable image features in said raw information, and to determinedistance information indicative of a distance between said image sensorand at least multiple points represented by said distinguishable imageinformation, and producing an output indicative of said pointinformation, representing a surface of the object being imaged.
 2. Anassembly as in claim 1, wherein said optical aperture includes first andsecond openings separated by a first distance, with respective colorfilters thereon, such that a first opening produces a first output thatis coded in a first color, and a second output aperture produces asecond output that is coded in a second color distinguishable from thefirst color, and where said processor analyzes said raw information torecognize points of said first color and said second color to determinesaid distance information based on said points of said first color andsaid points of said second color.
 3. An assembly as in claim 2, whereinsaid recognize multiple distinguishable features comprises finding saidfirst output in said first color, and said second output in said secondcolor, separated by a distance that is based on the first distancebetween said first and second openings on said optical aperture.
 4. Anassembly as in claim 2, wherein said raw information includesinformation processed by the image sensor without an image actuallybeing produced.
 5. An assembly as in claim 1, wherein said opticalaperture includes first and second openings separated by a firstdistance, with respective polarization filters thereon, such that afirst opening produces a first output that is coded in a firstpolarization, and a second output aperture produces a second output thatis coded in a second polarization distinguishable from the firstpolarization, and where said processor analyzes said raw information torecognize points of said first polarization and said second polarizationto determine said distance information based on said points of saidfirst polarization and said points of said second polarization.
 6. Anassembly as in claim 1, wherein said optical aperture includes at leastone aperture of a recognizable shape, and said distinguishable imageinformation distinguishes said recognizable shape.
 7. An assembly as inclaim 1, wherein said optical aperture includes an aperture that changesto have different characteristics at different times, and saiddistinguishable image is based on a specific time at which said image isproduced.
 8. An assembly as in claim 1, wherein said aperture iselectronically controlled.
 9. An assembly as in claim 1, wherein saidaperture is electronically controlled to have different characteristicsat different times.
 10. An assembly as in claim 1, further comprising arelay lens system in said optical train, adjusting a location of theoptical aperture to be positioned at the aperture plane.
 11. An assemblyas in claim 1, wherein said aperture is a single aperture having anelongated narrow slit aperture having a length which is longer than itis wide.
 12. An assembly as in claim 11, wherein said image informationhas at least one distinguishable image therein having a length relatedto said length.
 13. An assembly as in claim 12, wherein said processoruses said length to determine three dimensional information.
 14. Anassembly as in claim 11, wherein said aperture is placed such that acenter image of the object is visible through a central portion of saidaperture, and an outer two images are visible based on distal edges ofsaid aperture.
 15. An assembly as in claim 11, further comprising movingsaid aperture.
 16. An assembly as in claim 11, further comprisingcalibrating a relationship between a length of an image and a Z axisvalue corresponding to said length, and using said relationship todetermine Z axis information.